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" The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 



- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 



3) n Since this application is in con(jition for allowance except for fomnal matters, prosecution as to the merits is 

closecJ in accordance with the practice un(jer Exparfe Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) 13 Claim(s) 26-62 is/are pending in the application. 



Application Papers 

9)D The specification is objected to by the Examiner 

10)13 The drawing(s) filed on 03 March 2004 is/are: a)[3 accepted or b)n objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
11 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12)0 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)n All b)n Some * 0)0 None of: 

1 .□ Certified copies of the priority documents have been received. 

2.n Certified copies of the priority documents have been received in Application No. . 

sO Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Status 



1 )I3 Responsive to communication(s) filed on 03 March 2004 . 
2a)I3 This action is FINAL. 2b)n This action is non-final. 



4a) Of the above claim(s) is/are withdrawn from consideration. 



5) [3 Claim(s) 37 is/are allowed. 

6) 13 Claim(s) 26-30 and 32-61 is/are rejected. 

7) 13 Claim(s) 62 is/are objected to. 

8) 0 Claim(s) are subject to restriction and/or election requirement. 



Attachment(s) 

1 ) IS Notice of References Cited (PTO-892) 

2) CD Notice of Draftsperson's Patent Drawing Review (PTO-948) 

3) CD Information Disclosure Statement(s) (PTO-1449 or PTO/SB/08) 



4) CD interview Summary (PTO-413) 



5) CD Notice of Informal Patent Application (PTO-152) 

6) □ Other: . 



Paper No(s)/Mail Date. 



Paper No(s)/Mail Date 



U.S. Patent and TfBdemartt Office 

PTOL-326 (Rev. 1-04) 
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DETAILED ACTION 



Response to Amendment 



1 . The amendment and written response filed on March 03, 2004 has been entered. 

2. Claims 1-25 are cancelled. 

3. Claim 62 is added. 

4. Claims 26-62 are pending. 



5. Applicant's arguments filed on March 03, 2004 have been fully considered but they are 
not persuasive. 

The Tatsuhiko Reference: 

Applicant argues the Tatsuhiko reference does not teach "capturing an image of at least 
part of the surrounding area of the motor vehicle by a first optical opening of a camera device" 
and "capturing an image of at least part of the interior of the motor vehicle by a second optical 
opening of the camera device". In summary, applicant argues Tatsuhiko does not anticipate all 
the limitations recited in the claim. 

Upon further review of specification, and in light of applicant's arguments, the examiner 
respectfully disagrees for the following reasons. Tatsuhiko discloses a camera device mounted 
inside a vehicle, which rotates alternatively to capture images outside the vehicle as well as 
images inside the vehicle. Tatsuhiko does not teach such camera contains two optical openings. 
Nonetheless, it would have been an obvious matter of design choice to modify the Tatsuhiko 
reference by having two optical openings to capture images inside and outside the vehicle 
alternatively, since applicant has not disclosed two optical openings in a camera would have any 



Response to Arguments 
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particular advantages when it comes to taking images alternatively^ and it appears that having a 
camera that rotates to capture images alternatively would perform equally well 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

6. Claims 26-28, 32-39, 44-46, 51-58 & 59-61 are rejected under 35 U.S.C. 103(a) as being 

unpatentable over Tatsuhiko et al (Japanese Publication No. 10-35320, a copy of translation is 

attached). 

a. Referring to Claim 26, Tatsuhiko discloses (1) capturing an image of at least part 
of the surrounding area of the motor vehicle by a camera device (Tatsuhiko, in paragraph 
[0006] teaches using an image pick-up equipment to obtain pictures of surrounding area 
of vehicle); (2) capturing an image of at least part of the interior of the motor vehicle by 
the camera device (Tatsuhiko in paragraph [0007] teaches using the same image pick-up 
equipment to obtain pictures of vehicle interior, such as face or eyes of the operator of the 
vehicle as described in paragraph [0008]); the steps (1) and (2) being performed 
alternately (Tatsuhiko in paragraph [0023] teaches the image pick-up equipment can take 
pictures easily both outside and inside. In addition, Tatsuhiko in paragraph [0010] teaches 
a control panel is equipped with a mode transfer switch, which implies the steps 1 and 2 
are switched back and forth); and (3) transmitting the images obtained in steps (1) and (2) 
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to a processing unit (Tatsuhiko in paragraph [0010] discloses an image processing unit is 
used to process the pictures taken by the image pick-up equipment). Tatsuhiko discloses a 
camera device mounted inside a vehicle, which rotates alternatively to capture images 
outside the vehicle as well as images inside the vehicle. However, Tatsuhiko does not 
teach such camera contains two optical openings. Nonetheless, it would have been an 
obvious matter of design choice to modify the Tatsuhiko reference by having two optical 
openings to capture images inside and outside the vehicle alternatively, since applicant 
has not disclosed two optical openings in a camera would have any particular advantages 
when it comes to taking images alternatively, and it appears that having a camera that 
rotates to capture images alternatively would perform equally well. 

b. Referring to Claim 27, Tatsuhiko discloses wherein the at least part of the 
surrounding area of the vehicle is in a direction of travel (Tatsuhiko in paragraph [001 1] 
teaches "front recognition mode", which is in the direction of travel). 

c. Referring to Claim 28, Tatsuhiko discloses wherein the at least part of the interior 
of the vehicle includes parts of a body of a driver (Tatsuhiko in paragraph [0008] teaches 
the face of the operator of the vehicle) 

d. Referring to Claim 32, Tatsuhiko discloses only an image of an area surrounding 
the motor vehicle visible to the camera device is captured in the step (1); and only an 
image of the interior of the motor vehicle visible to the camera device is captured in the 
step (2) (see paragraphs [0006] and [0007]) 

e. Referring to Claim 33, Tatsuhiko discloses wherein switching back and forth 
between the step (1) and the step (2) is accomplished via at least one light valve 
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(Tatsuhiko in paragraph [0010] teaches switching back and forth between the step 1 and 2 
is accomplished via at least one light valve). 

f. Referring to Claim 34, Tatsuhiko discloses wherein the at least one light valve is 
an electro-optical light valve (an camera inherently contains at least one electro-optical 
light valve). 

g. Referring to Claim 35, Tatsuhiko discloses switching back and forth between 
capturing a partial area of the interior and a partial area of the surrounding area; 
processing by the processing unit the partial areas captured; and capturing a next partial 
area (the examiner already explains in Claim 26 above that the image pick-up equipment 
switches back and forth to process two or more modes at the same time. Therefore, the 
image pick-up equipment must sometimes capture a partial area of the interior and a 
partial area of the surrounding area). 

h. Referring to Claim 36, Tatsuhiko discloses capturing a face of a driver, the face 
including eyes of the driver (paragraph [0008]). 

i. Referring to Claim 37, Tatsuhiko discloses capturing at least one of road 
markings and a position of the vehicle relative to the road markings (see paragraph 
[0018]) 

j. Referring to Claim 38, Tatsuhiko discloses evaluating at least one of the face of 
the driver and a position of the vehicle relative to road markings to determine at least one 
of whether the eyes of the driver are open and whether the vehicle is moving beyond a 
predefined area of the road markings; and issuing at least one of a visual warning an 
audio warning based on the evaluation (paragraphs [0015], [0018], [0062], and [0095]). 
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k. Referring to Claim 39, Tatsuhiko discloses capturing road signs (when the image 
pick-up equipment takes pictures of surrounding area of the vehicle, the road signs will 
inherently be captured). 

I With regard to Claim 44, all limitations are addressed in Claim 26 (note when the 
camera captures the image outside, it is the first beam path; when it captures the image 
inside, it is the second beam path). 

m. With regard to Claim 45, see explanation in claims 27 and 28. 
n. With regard to Claim 46, see explanation in Claim 28. 
o. With regard to Claim 51, see explanation in Claim 33. 

p. Referring to Claim 52, every camera has a light valve, which is a liquid crystal 
cell. 

q. Referring to Claim 53, every camera has at least one deviation mirror, 
r. Referring to Claim 54, every camera has at least one mirror, which is semi- 
transparent. 

s. Referring to Claim 55, everything camera has at least one deviation which is 
concave or convex. 

t. Referring to Claim 56, Tatsuhiko discloses the camera device has a single camera, 
u. Referring to Claim 57, Tatsuhiko discloses the single camera is one of a CCD 
camera and a CMOS camera (see paragraph [0031]). 

V. Referring to Claim 59, Tatsuhiko discloses at least one of visual output units and 
acoustic output units connected to the processing unit, the at least one of visual output 
units and acoustic output units configured to issue a warning to a driver when one of eyes 
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of the driver are closed and the vehicle is about to move beyond a marked area of a road 
(see paragraph [0079]). 

w. Referring to Claim 60, Tatsuhiko discloses the camera device is one of arranged 
in an upper part of a windshield and integrated into a roof of the vehicle (see figure 2) 
X. Referring to Claim 61, Tatsuhiko discloses an adjustment device configured to 
adjust the at least one deviation mirror so that at least eyes and lips of a driver can be 
seen in the image of the interior of the vehicle captured by the camera device (Tatsuhiko 
in paragraph [0026] teaches changeover switch 105 performs the function of changing 
focus when the camera starts capturing interior images. Focus change inherently adjusts 
the deviation mirror inside the camera. In addition, the examiner explains before 
Tatsuhiko teaches taking images of an operator's face). 
7. Claims 29-30 and 47-50 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Tatsuhiko in view of Ueno et al (U.S. Patent No. 5,293,427). All the arguments and applicability 
in Claims 26 and 44 are incorporated herein. 

a. Referring to Claim 29, Tatsuhiko discloses illuminating the interior of the vehicle. 
However, Tatsuhiko does not explicitly teach illuminating the interior by a 
radiation source, the radiation source emitting a radiation at least substantially 
invisible to the human eye. Ueno at column 2, lines 50-61, teaches use of the 
infrared rays, which is substantially invisible to the human eyes. At the time the 
invention was made, it would have been obvious to a person of ordinary skill in 
the art to take face images by a camera in synchronism with the infrared rays. One 
of ordinary skill in the art would have been motivated to do this because 



Application/Control Number: 09/743,305 Page 8 

Art Unit: 2621 

Tatsuhiko in paragraph [0008] teaches taking picture of a vehicle operator to 
determine whether the operator is dozing. And Ueno at column 4, lines 24-36, 
teaches using a camera taking images of a driver for the same purpose with an 
additional feature of infrared rays, which activates the camera to take an image of 
the driver to determine if the driver is dozing. Therefore, it would be reasonable to 
a person of ordinary skill in the art to add an infrared source to Tatsuhiko 's 
system for camera activation purpose. 

b. Referring to Claim 30, Ueno teaches the radiation source is an infrared radiation 
source. 

c. With regard to Claim 47, see explanation in Claim 29. 

d. With regard to Claim 48, see explanation in Claim 30. 

e. With regard to Claim 49, see explanation in Claim 29. 
f With regard to Claim 50, see figure 1 in Ueno. 

8. Claims 40-41 are rejected under 35 U.S.C. 103(a) as being unpatentable over Tatsuhiko 
in view of Breed et al (U.S. Patent No. 5,845,000). All the arguments and applicability in Claim 
26 are incorporated herein. 

a. Referring to Claim 40, Tatsuhiko discloses taking pictures of a vehicle's interior. 
However, Tatsuhiko does not teach using such picture images to determine at 
least one of a number of people in the vehicle and seat occupancy. Breed in the 
abstract, teaches using the image data obtained through a camera to determine the 
position of the vehicle occupant relative to the airbag, and disable deployment of 
the airbag if necessary. At the time the invention was made, it would have been 
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obvious to a person ordinary skill in the art to take image data of a vehicle interior 
to determine at least one of a number of people in the vehicle and seat occupancy. 
One of ordinary skill in the art would have been motivated to do this because 
Breed in abstract teaches determining the position of the vehicle occupant through 
the image taken by a camera, which is equipped in Tatsuhiko's system as well, 
would be an advantage that allows the system to disable deployment of the airbag 
if the occupant is positioned so that he/she is likely to be injured by the 
deployment of the airbag. 
b. Referring to Claim 4 1 , see explanation with regard to deactivating an airbag in 
Claim 40, also see figure 2 in Breed. 
9. Claim 42-43 are rejected under 35 U.S.C. 103(a) as being unpatentable over Tatsuhiko in 
view of Kondo (U.S. Patent No. 6,396,954). All the arguments and applicability in Claim 26 are 
incorporated herein. 

a. Referring to Claim 42, Tatsuhiko discloses taking pictures of a driver's face. 
However, Tatsuhiko does not explicitly teach capturing lip movements of a 
person in the vehicle to support a speech input system. Kondo at column 6, lines 
38-46, teaches use of a CCD camera in accordance with speech input system. At 
the time the invention was made, it would have been obvious to a person of 
ordinary skill in the art to use a camera to capture the movement of a driver's 
mouth to support a speech input system because as Kondo points out at column 6, 
line 45-46, it will increase the rate of speech recognition. 

b. Referring to Claim 43, Tatsuhiko discloses the person is a driver of the vehicle. 
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10. Claim 58 is rejected under 35 U.S.C. 103(a) as being unpatentable over Tatsuhiko in 
view of Diner (U.S. Patent No. 5,065,236). All the arguments and applicability in Claim 44 are 
incorporated herein. 

a. Referring to Claim 58, Tatsuhiko does not expUcitly disclose the image pick-up 
equipment contains 2 cameras for capturing images stereoscopically. Diner at 
column 2, lines 30-54, using two cameras to produce stereoscopic images. At the 
time the invention was made, one of ordinary skill in the art would have been 
motivated to do this because Diner at column 2, lines 48-49, teaches parallel 
cameras with shifted and magnified stereoscopic images can provide fUlly 
undistorted views of objects with control of the perceived depth of the plane of 
interest. 

Allowable Subject Matter 

1 1 . Claim 3 1 is allowed. 

The following is a statement of reasons for the indication of allowable subject matter: 
Independent Claim 31 defines features of superimposing the at least part of the interior of 
the vehicle visible to the camera device on the at least part of the surrounding area of the vehicle 
visible to the camera device; and determining the image of the at least part of the interior of the 
vehicle by subtracting the image of the at least part of the surrounding area. These features in 
combination with other features in Claim 31, which is the broadest allowable claim, are not 
taught or suggested by the art of record. 
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12. Claim 64 is objected to as being dependent upon a rejected base claim, but would be 
allowable if rewritten in independent form including all of the limitations of the base claim and 
any intervening claims. 

Conclusion 

13. The prior art made of record and not relied upon is considered pertinent to appHcant's 
disclosure. 

a. Park et al, U.S. Patent No. 6,690,374 B2, see figure IB and abstract. 

b, Gilber et al, U.S. Patent No. 6,654,019 B2, see figures 5A and 5B. 

14. Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1. 136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

15. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Tom Y Lu whose telephone number is (703) 306-4057. The 
examiner can normally be reached on 8:30AM-5PM. 



• # 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Leo H Boudreau can be reached on (703) 305-4706. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 



Tom Y. Lu 




TECHNOLOGY CENTER 2600 



